chore(ci): migrate Python dependency locking from pip-tools to uv by yogarajalakshmi-s · Pull Request #3541 · kubeflow/trainer

yogarajalakshmi-s · 2026-05-24T20:16:47Z

What this PR does / why we need it

Migrates Python dependency locking from pip-tools (pip-compile) to uv, aligning kubeflow/trainer with kubeflow/sdk tooling.

Add pyproject.toml for dataset and model initializers
Replace requirements-lock.txt with uv.lock for all 3 components
Update osv-scanner.yaml to use uv lock --upgrade-package
Update validate-lockfile.yaml to use uv lock --check
Add helper scripts adapted from kubeflow/sdk
Remove hack/scripts/update-source-dep.py

Testing

uv lock --check passes for all 3 components locally
osv-scanner reads all 3 uv.lock files successfully (didn't get any issues)
Full workflow testing will happen in CI after /ok-to-test is applied

Note

All changes were written, reviewed and tested locally by me. (AI tooling was used to understand the codebase structure)

Which issue this PR fixes:
Part of #3535

Checklist:

Docs included if any changes are user facing

google-oss-prow · 2026-05-24T20:16:54Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tenzen-y for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

github-actions · 2026-05-24T20:16:57Z

🎉 Welcome to the Kubeflow Trainer! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards.
Our team will review your PR soon! cc @kubeflow/kubeflow-trainer-team

Join the community:

Slack: Join our #kubeflow-trainer Slack channel.
Meetings: Attend the Kubeflow AutoML and Training Working Group bi-weekly meetings.

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏

Copilot

Pull request overview

Migrates Python dependency locking in the repo from pip-tools (pip-compile-generated requirements-lock.txt) to uv (uv.lock), and updates CI workflows (lockfile validation + OSV auto-fix) to operate on uv.lock across the dataset initializer, model initializer, and Python API components.

Changes:

Added pyproject.toml + uv.lock for the dataset/model initializers and replaced existing requirements-lock.txt lockfiles with uv.lock across all Python components.
Updated GitHub workflows to validate lockfiles via uv lock --check and run OSV scanning/fixing against uv.lock.
Added helper scripts for OSV auto-fix flows and removed the legacy hack/scripts/update-source-dep.py.

Reviewed changes

Copilot reviewed 11 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`.github/workflows/validate-lockfile.yaml`	Switches lockfile sync validation from `pip-compile` output comparison to `uv lock --check`.
`.github/workflows/osv-scanner.yaml`	Updates Python OSV scan inputs to `uv.lock` and rewires the auto-fix logic to use `uv lock --upgrade-package` + optional overrides.
`.github/scripts/update_overrides.py`	Adds a script to manage `[tool.uv].override-dependencies` entries in `pyproject.toml`.
`.github/scripts/extract_version.py`	Parses `uv tree` output to extract an upgraded package version.
`.github/scripts/compare_versions.py`	Compares versions (PEP 440) to decide whether an override is needed.
`cmd/initializers/dataset/pyproject.toml`	Introduces uv-compatible dependency declaration for dataset initializer.
`cmd/initializers/dataset/uv.lock`	New uv lockfile for dataset initializer.
`cmd/initializers/dataset/requirements-lock.txt`	Removed pip-tools lockfile.
`cmd/initializers/model/pyproject.toml`	Introduces uv-compatible dependency declaration for model initializer.
`cmd/initializers/model/uv.lock`	New uv lockfile for model initializer.
`cmd/initializers/model/requirements-lock.txt`	Removed pip-tools lockfile.
`api/python_api/uv.lock`	New uv lockfile for Python API.
`api/python_api/requirements-lock.txt`	Removed pip-tools lockfile.
`hack/scripts/update-source-dep.py`	Removes the legacy dependency-updater used by the old pip-tools OSV auto-fix flow.

Fiona-Waters

Thanks for this contribution @yogarajalakshmi-s
Looks good so far. One thing I notice is a missing test file. The kubeflow/sdk repo includes a test_scripts.py with pytest coverage for the helper scripts. Since the scripts in this PR are adapted from there, it would be worth bringing over the test file too — these run in a nightly security workflow where silent failures are hard to notice.

yogarajalakshmi-s · 2026-05-25T12:12:35Z

Thanks for this contribution @yogarajalakshmi-s Looks good so far. One thing I notice is a missing test file. The kubeflow/sdk repo includes a test_scripts.py with pytest coverage for the helper scripts. Since the scripts in this PR are adapted from there, it would be worth bringing over the test file too — these run in a nightly security workflow where silent failures are hard to notice.

Thank you for your review, @Fiona-Waters! I will include the test script.

andreyvelich · 2026-05-26T00:44:00Z

/ok-to-test

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

…packaging dependency Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

Copilot AI review requested due to automatic review settings May 24, 2026 20:16

google-oss-prow Bot requested review from jinchihe and kuizhiqing May 24, 2026 20:16

google-oss-prow Bot added the size/XXL label May 24, 2026

Copilot started reviewing on behalf of yogarajalakshmi-s May 24, 2026 20:17 View session

Copilot AI reviewed May 24, 2026

View reviewed changes

yogarajalakshmi-s mentioned this pull request May 25, 2026

update_overrides.py drops Security overrides header on repeated runs kubeflow/sdk#502

Closed

Fiona-Waters reviewed May 25, 2026

View reviewed changes

google-oss-prow Bot added the ok-to-test label May 26, 2026

yogarajalakshmi-s added 8 commits May 27, 2026 23:03

chore(ci): add pyproject.toml for dataset and model initializers

4c01875

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): add uv.lock lockfiles for Python initializers and api

593ff41

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): add uv helper scripts adapted from kubeflow/sdk

eb6fa87

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): migrate osv-scanner and validate-lockfile to use uv

1edd7b5

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): fix header comment loss in update_overrides on repeated runs

e096f75

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): add test coverage for uv helper scripts

ca4081f

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): fix version stripping, case-insensitive pkg detection and …

9b449d3

…packaging dependency Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

chore(ci): add missing pyproject.toml to add-paths and fix trigger paths

7158474

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

yogarajalakshmi-s force-pushed the chore/migrate-pip-tools-to-uv branch from 0cabac4 to 7158474 Compare May 28, 2026 03:05

yogarajalakshmi-s added 2 commits June 4, 2026 12:03

resolve merge conflict by removing update-source-dep.py

4158fa8

add boilerplate headers to .github/scripts

039cc26

Signed-off-by: Yogarajalakshmi S <yogarajalakshmis@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(ci): migrate Python dependency locking from pip-tools to uv#3541

chore(ci): migrate Python dependency locking from pip-tools to uv#3541
yogarajalakshmi-s wants to merge 10 commits into
kubeflow:masterfrom
yogarajalakshmi-s:chore/migrate-pip-tools-to-uv

yogarajalakshmi-s commented May 24, 2026

Uh oh!

google-oss-prow Bot commented May 24, 2026

Uh oh!

github-actions Bot commented May 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fiona-Waters left a comment

Uh oh!

yogarajalakshmi-s commented May 25, 2026

Uh oh!

andreyvelich commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

yogarajalakshmi-s commented May 24, 2026

What this PR does / why we need it

Testing

Note

Uh oh!

google-oss-prow Bot commented May 24, 2026

Uh oh!

github-actions Bot commented May 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fiona-Waters left a comment

Choose a reason for hiding this comment

Uh oh!

yogarajalakshmi-s commented May 25, 2026

Uh oh!

andreyvelich commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants